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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

I) ^ Responsive to communication(s) filed on 20 April 2004 . 

2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) E3 Claim(s) 1-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-26 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 20 April 2004 is/are: a)S accepted or b)D objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

II) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
* Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1, 3 - 9, 13, 15, 17, 18, 20, 22, 23, 25 and 27 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Sharma IEEE pub titled: "Bayesian sensor 
image fusion using local linear generative models" in view of Matsugu (US pub no 
2003/0044073) and Hoffberg (US pat no 5,867,386). 

With regards to claim 1, Sharma discloses a vision fused integrated vision system 
comprising: 

a non-HVS sensor array for providing a sensor output from each sensor in the 
array (see figure 1, shows images taken an airport run way) and 

a feature extraction mechanism for extracting multi-resolution features of an 
objective (see table 1 , resolutions are broken down into seven levels) and for forming a 
single, fused feature image of the objective the sensor outputs (see image in figure 6c, 
is the fused image of "Averaging" and "Selection" image). 

Sharma does not disclose registration mechanism and an association engine. Matsugu 
discloses 
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a database of images of an objective stored in a memory (see figure 1a, element 

A4); 

a registration mechanism for comparing the extracted fused, feature image to a 
database of expected features of the objective and for providing registered sensor 
output vectors (see figure 1B, S15); 

an association engine for processing the registered sensor output vectors with 
the database of objective images; including an associative match mechanism for 
comparing the registered sensor output vectors to said database of images of the 
objective image (see paragraph 43, last sentence, the match results with the least error 
is associated with input image; also see figure 14, are examples of storage format of 
feature vectors stored in database); and 

a monitor for displaying any result image areas extracted (see figure 1 A, A9). 
One skilled in the art would include in matching feature vectors to identifying objects in 
an image because the size of feature vectors are smaller an image matrix, which save 
time on processing. 

Sharma and Matsugu do not disclose a HVS display for displaying a HVS perceptible 
image from the database objective images. Hoffberg discloses a HVS display for 
displaying a HVS perceptible image from the database objective images (see column 
31 , lines 44 to 50). One skilled in the art would include HVS display because the user 
can change colors at desired locations, which improves image quality. 
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With regards to claim 3, Sharma discloses a fused feature image is formed by vector 
addition of sensor outputs (see equation 7, the sensor values are added). 

With regards to claim 4, Matsugu discloses feature extraction mechanism includes V1 
feature detection and K-WTA processing (see image in figure 3 shows a human face 
which is biological). 

With regards to claim 5, Matsugu further discloses said associative match mechanism 
includes a best match mechanism (see paragraph 43, last sentence, the image stored 
in the database with the least error in matching is the best match image). 

With regards to claim 6, Matsugu further discloses said associative match mechanism 
includes an exact match mechanism (see paragraph 43, last sentence, the image 
stored in the database with the least error in matching is an exact match image). 

With regards to claim 7, Hoffberg discloses HVS display device (column 31 , lines 44 to 
50) and Matsugu discloses comparison vector points to an image of an objective in said 
database after said exact match mechanism locates an exact match between a fused 
feature image and an image of an objective (see figure 1B, S15). 



Application/Control Number: Page 5 

10/828,742 

Art Unit: 2624 

With regards to claim 8, Matsugu discloses input for said match mechanism is output 
from a best match mechanism (see figure 1 A the monitor is an output device and the 
camera is an input device). 

With regards to claim 9, Matsugu discloses a registration mechanism normalizes a 
feature image of the objective across sensor modalities (see paragraph 63). 

With regards to claim 13, see the rationale and rejection for claim 1 . 

With regards to claim 15, see the rationale and rejection for claim 9. 

With regards to claim 17, see the rationale and rejection for claim 4. 

With regards to claim 18, Matsugu discloses registering the extracted features with 
known features of the objective to provide registered sensor output vectors includes 
comparing extracted features with known features of a generic representation of a class 
of similar objectives (see figure 2, features such as intersection, curves arrow type 
intersection, circle are classified). 

With regards to claim 20, Matsugu discloses processing using edge extraction (see 
paragraph 43, lines 10 to 15). 
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With regards to claim 22, see the rationale and rejection for claim 3. 

With regards to claim 23, see the rationale and rejection for claim 5. 

With regards to claim 25, see the rationale and rejection for claim 7. 

With regards to claim 27, see the rationale and rejection for claim 8. 

3. Claims 2 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Sharma in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 1 further in 
view of Abernathy (US pat no 7,298,869). 

With regards to claim 2, Sharma, Matsugu and Hoffberg disclose all the limitations in 
claim 1 , but do not disclose sensor array includes a LWIR sensor, a SWIR sensor and a 
MMW sensor. Abernathy discloses sensor array includes a LWIR sensor, a SWIR 
sensor and a MMW sensor (see column 2, lines 1 to 1 1 and column 4, lines 45 to 51). 
One skilled in the art would include sensor array includes a LWIR sensor, a SWIR 
sensor and a MMW sensor because each sensor extracts features more sensitive to it 
respective spectrum range, which extracts clearer details. 

With regards to claim 14, see the rationale and rejection for claim 2. 
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4. Claims 10 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sharma in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 1 further 
in view of Hoffman (US pat no 6,035,057). 

With regards to claim 10, Sharma, Matsugu and Hoffberg disclose all the limitations in 
claim 1, but do not disclose Voronoi classifier. Hoffman discloses operation of a Voronoi 
classifier for training the association engine with an enhanced feature image (see 
column 15, lines 36 to 65 and figure 1). One skilled in the art would include Voronoi 
classification because this method isolate region of interest and extracts detail from 
those region, which improve image quality. 

With regards to claim 16, see the rationale and rejection for claim 10. 

5. Claims 1 1 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sharma in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 1 further 
in view of Yoshioka (US pat no 6,035,053). 

With regards to claim 11, Sharma, Matsugu and Hoffberg disclose all the limitations in 
claim 1, but do not disclose a hazard detection mechanism. Yoshioka discloses hazard 
detection mechanism for comparing the registered sensor output vector to a best match 
comparison of the output vector to the objective image database to identify possible 
comparison of the objective by a hazardous entity (see column 5, lines 1 to 17 and 
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figure 2, the obstacles are read as hazardous entity). One skilled in the art would 
include hazardous entity because to avoid cars getting into accidents. 

With regards to claim 24, see the rationale and rejection for claim 11. 

6. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Sharma 
in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 1 further in view of 
Dolfing (US pat no 6,421 ,640). 

With regards to claim 12, Sharma, Matsugu and Hoffberg disclose all the limitations of 
claim 1 , but do not disclose measuring confidence using entropy as a heuristic measure 
of system integrity. Dolfing discloses measuring confidence using entropy as a heuristic 
measure of system integrity (see column 1 , lines 37 to 41). One skilled in the art would 
include measuring the confidence of result of an identification process because to 
determine which template matches best with the input image, to improve accuracy of 
the recognition process. 

7. Claims 19 and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sharma in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 1 further 
in view of Hutcheson (US pat no 5,161 ,204). 
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With regards to claim 19, Sharma, Matsugu and Hoffberg disclose all the limitations of 
claim 13, but do not disclose output vectors in an association engine, which includes 
processing by a neutral network. Hutcheson discloses output vectors in an association 
engine, which includes processing by a neutral network (see figure 14). One skilled in 
the art would include step of using neutral network because it determines the best set of 
weights to identify a feature, which improves feature details extraction. 

With regards to claim 21, see the rationale and rejection for claim 19 (diagram in figure 
14 has parallel processing). 

8. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Sharma 
in view of Matsugu ('073) and Hoffberg ('386) as applied to claim 25 further in view of 
official notice (MPEP 2144.03). 

With regards to claim 26, Sharma, Matsugu and Hoffberg disclose all the limitations in 
claim 25, but do not disclose displaying an image selected from the database of 
objective images as indicated by the pointer. However, it is well known in the art to 
output the most desirable signal or image to be display to the operator or user. One 
skilled in the art would include selected image indicated by a pointer because the 
operator may decide which whether the outputted image is the best image, improving 
identification accuracy. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alex Liew whose telephone number is (571)272-8623. 
The examiner can normally be reached on 9:30AM - 7:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella can be reached on (571) 272-7778. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Alex Liew 

AU2624 

12/4/07 
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SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



